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Abstract. We discuss a new notion of pattern avoidance moti- 
vated by the operad theory: pattern avoidance in planar labelled 
trees. It is a generalisation of various types of consecutive pattern 
avoidance studied before: consecutive patterns in words, permu- 
tations, coloured permutations etc. The notion of Wilf equivalence 
for patterns in permutations admits a straightforward generalisa- 
tion for (sets of) tree patterns; we describe classes for trees with 
small numbers of leaves, and give several bijections between trees 
avoiding pattern sets from the same class. We also explain a few 
general results for tree pattern avoidance, both for the exact and 
the asymptotic enumeration. 



1. Introduction 

For pattern avoidance in words, apart from the "real word interpre- 
tation" (enumerate words not containing any obscene subwords), the 
pattern avoidance problem arises from studying (noncommutative) 
algebras with monomial relations. For example, describing words 
in the alphabet {A, B, . . . , Z} not containing the word FCUK as a sub- 
word is equivalent to figuring out which monomials in generators 
%i, ...,X26 form a basis in the algebra k(xi, . . . , Xje I X6X3X21XU = 0). 
The significance of algebras with monomial relations is, in turn, ex- 
plained by the theory of Grobner bases which gives a method of 
finding a "monomial replacement" for every algebra with monomial 
relations [40]. Similarly, consecutive pattern avoidance in permu- 
tations [2, 3] and coloured permutations [29] can be interpreted in 
terms of shuffle algebras with monomial relations [9]. The goal of 
this paper is to introduce to the combinatorics audience a new no- 
tion of pattern avoidance naturally arising when studying operads. 
Operads are similar to associative algebras, but while associative al- 
gebras and groups capture the kind of associativity that one observes 
when composing transformations of some set, operads capture the 
associativity exhibited when composing operations with several ar- 
guments. The property of an operation of having more than one 
argument results in a choice that is not present in the choice of alge- 
bras: one may either assume that our operations do not possess any 
symmetries or allow symmetries in the picture. In the former case, 
the pattern avoidance in question is the pattern avoidance in planar 
rooted trees; a few papers, both of combinatorial spirit [1, 33, 36, 13] 
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and operads-inspired [28] have been dealing with the arising ques- 
tions of enumeration. In the case of operations with symmetries, the 
corresponding notion has not been studied before, and this paper 
is an attempt to give an elementary introduction to the arising re- 
search area. The "right" notion of a monomial relation for operations 
with symmetries is not as obvious as one might think: the action of 
symmetries makes every relation have too many consequences, and 
the arising class of "operads with monomial relations" appears to be 
way too narrow to be truly interesting or useful. The way to define 
monomial relations which avoids narrowing things down, which in 
particular led to a theory of operadic Grobner bases, was suggested 
in [8]; the corresponding algebraic object is called a shuffle operad. In 
this paper we, however, shall try to concentrate on the combinatorial 
aspects of the subject, touching the algebraic aspects only briefly; for 
details on the algebraic aspects, see, for example, [6, 7, 8]. Though 
we attempt to keep this article relatively self-contained, familiarity 
with the key results of the theory of consecutive pattern avoidance 
in permutations could be useful; for relevant historical information 
on pattern avoidance as well as the state-of-art of this area the reader 
is referred to a recent monograph [24]. 

There are several types of questions in the theory of tree patterns 
which are meaningful from the operadic viewpoint. First of all, for 
a given set of patterns, exact enumeration results for trees avoiding 
that set are very important. Some examples of that sort appear in 
the following sections; in many cases the corresponding sequences of 
numbers are well known in combinatorics, but in many other cases 
one ends up with a sequences that appear to be unrelated to classical 
enumeration problems. Second, there is a question of asymptotic 
enumeration. We shall prove some results of that sort relying on 
the Golod-Shafarevich technique [17] (which has recently been re- 
discovered in relation to combinatorics of pattern avoidance [4, 5, 
35]). Also, there is a question of recognising the class of generating 
functions arising in this type of enumerating questions. However, 
while for word avoidance the answer is that the generating functions 
arising as answers for enumerating the avoidance of finite sets of 
words are always rational, even for consecutive pattern avoidance 
in permutations the class of arising generating functions does not 
have a satisfactory description. The answer is known if one ignores 
the leaf labels completely; in that case, the generating functions are 
algebraic (as proved in [36] in the case of binary trees, in [13] in 
the case of ternary trees, and in [22] in the general case). For the 
notion of tree pattern avoidance discussed here, it follows from one 
of the theorems of [22] that under some additional assumptions on 
the set of patterns ("shuffle regularity") the generating functions for 
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tree pattern avoidance are differentially algebraic (i.e. satisfy non- 
linear differential equations with polynomial coefficients). Finally it 
is interesting to enumerate the Wilf equivalence classes of sets of tree 
patterns 1 , and we discuss some basic results of that sort. 

This paper is organised as follows. In Section 2, we define planar 
labelled rooted trees and tree patterns, and show that this notion 
includes the classical types of consecutive pattern avoidance as par- 
ticular cases. In Section 3, we present some results on asymptotics for 
tree pattern avoidance, and introduce the notion of growth rate for a 
given set of patterns. In Section 4, we formulate an exact enumeration 
result on tree pattern avoidance which follows from our work with 
Khoroshkin [7], and discuss some consequences of that result and 
related questions. Finally, in Section 5 we discuss various examples 
of patterns with small numbers of leaves. Each of these sections also 
contains some conjectures and natural questions that are beyond the 
scope of this paper. We also included an appendix explaining how 
pattern avoidance in trees arises in the operadic context. 

Acknowledgements. I wish to thank Anton Khoroshkin and Dmitri 
Piontkovski for sending me a copy of their forthcoming preprint [22]. 

2. Planar labelled tree patterns 

2.1. Tree patterns. Trees have vertices and edges. A rooted tree is a 
tree with a distinguished vertex, called the root. A rooted tree can 
be directed "away from the root"; this way every vertex except for 
the root has exactly one parent. Vertices whose common parent is a 
given vertex v are called children of v. Vertices that have at least one 
child are called internal, vertices with no children are called leaves. A 
planar rooted tree is a rooted tree together with a total order on the 
set of children of each vertex, we shall think of it as of embedded in 
plane, the children of each vertex placed in the increasing order from 
the left to the right. 

Throughout the paper, X = |_| n >2 X„ is a finite alphabet represented 
as a disjoint union of its subsets X n , n > 2. 

Definition 1. A planar X-labelled rooted tree is a planar rooted tree T 
with no internal vertices having exactly one child, and with a la- 
belling of all vertices fulfilling the following restrictions: 

- every internal vertex v with m children is labelled by an ele- 
ment x v eX m ; 

- every leaf of T is labelled by a positive integer in such a way 
that the following two conditions are satisfied: 



Equivalence classes for "strong equivalence" might lead to interesting combi- 
natorial results as well, but are much less natural from the operad point of view. 
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(1) (labelling set condition) the leaf labels are in one-to-one 
correspondence with the set [I] = {1,2, ...,1} (where I is 
the number of leaves of T); 

(2) (local increasing condition) if we temporarily assign to 
each internal vertex v the smallest of the labels of leaves 
that are descendants of v in T (thus every vertex of T, 
should it be an internal vertex or a leaf, has an integer 
assigned to it), then for each internal vertex i the integers 
assigned to its children increase from the left to the right. 

Notation: for i > 0, / > 1 we denote by X7~y(X) the set of planar 
X-labelled rooted trees with i internal vertices and I leaves, 

(1) £T l (X) = \j£T i , 1 (X) 

i>0 

is the set of planar X-labelled rooted trees with I leaves, 

(2) £T(X) = (J £Ty(X) 

!>0,/>l 

is the set of all planar X-labelled rooted trees. In particular, £T 0/ i(X) 
is non-empty only for I = 1 (and in that case it consists of exactly one 
element, a one-vertex tree with no edges), and XT~i(X) = £T 0/ i(X) 
(since every internal vertex has at least two children, hence in the 
presence of an internal vertex we end up with at least two leaves). 

Example 1. Let X = X 2 = {o, •}. The following tree is in £T 7i8 (X): 



1 3 2 7 4 6 




A tree T is said to be a left (right) comb if for every internal vertex 
of T only its leftmost (rightmost) child may not be a leaf. Note that 
the local increasing condition makes the notions of "left" and "right" 
very different: for example, if a planar rooted tree f is a left comb, 
the only restriction on the leaf labels is that the leftmost leaf of t is 
labelled by 1; by contrast, if t is a right comb, there is only one leaf 
labelling satisfying the local increasing condition. 

Most of the time throughout the paper we consider only the case 
X = X d , thus assuming that for our trees all internal vertices have 
the same number of children (that is, d children). This assumption is 
mostly technical (it allows for closed formulas in various statements), 
in particular, all the asymptotic results we prove and conjecture in 
Section 3 are expected to be true in full generality. One particular 
simplification is that for such trees the number of internal vertices and 
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the number of leaves are related: every tree with k internal vertices 
has kd-k + 1 leaves. 

Let us prove a basic enumerative result which will be useful later. 

Proposition 1. For X = X d , we have 
(4) \£T M - k+1 (X)\ = \X d \ 



{d\f ■ k\ ' 




Proof. It would be useful for our purposes to consider, along with 
planar X-labelled rooted trees, two other types of trees. We denote 

by T n {X) the set of planar rooted trees with n leaves whose internal 
vertices are labelled by X, and by T n {X) the set of planar rooted trees 
with n leaves whose internal vertices are labelled by X, and whose 
leaves are labelled by {1, 2, ... , n) (in all possible ways). 
Recall that for X = X d = {•}, we have [39] 

(5) \T kd . k+1 (X)\ = 

For the general case, we note that \T kd - k+ i(X)\ is \X d \ k times larger, 
since every of k internal vertices should acquire a label from X d . 
Also, it is clear that 

(6) \£T kd . k+ i(X)\ = -^- k \T M - k+l {X)\, 

since for each of the k internal vertices of a tree only one of the 
d\ permutations of its subtrees fulfils the local increasing condition. 
Finally, 

(7) \T M - k+ i(X)\ = (kd-k + l)!|7V fc+1 (X)|, 

since all leaf labelling are allowed in T kd - k+ \{X), and the formula 
follows. □ 

By a subtree of a planar labelled rooted tree T we always mean a 
subtree S with its root at one of the internal vertices of T such that for 
each internal vertex of S all its children in T are also its children in S. 
(This way we can guarantee that the labels of internal vertices make 
sense for S.) Note that the second condition on leaf-labelling assigns 
to each internal vertex of a tree T a temporary integer label, so that a 
subtree S of a tree T almost belongs to X7~(X): its leaves are labelled 
by integers such that the local increasing condition is satisfied (but 
the labelling set condition might not be satisfied). Replacing, for a 
subtree S with / leaves, its leaf labels by 1, 2, . . . , I in the unique order- 
preserving way, we shall obtain a tree st(S) e £T{X) which we call 
the standardisation of S. 
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Definition 2. A tree T e £T{X) is said to contain a tree P 6 £T(X) as 
a pattern if there exists a subtree S of T for which P = st(S). Otherwise 
T is said to avoid S. 

Example 2. Let us recall the tree from Example 1, and consider its 
subtree represented by thick lines in the following figure: 

1 3 2 7 4 6 




(8) 

15 2 4 

Y V 

This subtree acquires the leaf numbering \/ , and after standard- 

1423 1423 

Y Y Y V 

isation we get \/ . So \/ occurs in our tree as a pattern. 



Throughout the paper, we only consider one type of tree patterns, 
so we often use the words "tree pattern" where one should say "pla- 
nar labelled rooted tree pattern"; we hope that the reader will appre- 
ciate this attempt to abbreviate things. 

Let us fix some set of labels X, and consider the pattern avoidance 
for planar X-labelled trees. The central question arising in the theory 
of pattern avoidance is that of enumeration of objects that avoid the 
given set of forbidden patterns 2? or, more generally, that contain 
exactly d occurrences of patterns from This question naturally 
leads to the following equivalence relations for tree patterns. Two 
sets of tree patterns 2? , 2?' c £fT(X) are said to be Wilf equivalent 
(notation: 2* ~ w if for every I, the number of ^-avoiding trees 
with I leaves is equal to the number of ^"-avoiding trees with I 
leaves. (In the case of (non-consecutive) permutation patterns the 
same notion was introduced by Wilf [42].) More generally, 2? and 
2^' are said to be (strongly) equivalent (notation: ~ 2 s ') if for 
every I and every k > 0, the number of trees with I leaves that have k 
occurrences of patterns from & is equal to the number of trees with 
I leaves that have k occurrences of patterns from . 

For enumeration, we shall primarily use the exponential generat- 
ing functions with respect to the number of leaves in trees, so that, 
for example, the generating functions for the label set X and the 

pattern set & are f x (z) = £ ^f- and >(z) = £ ^ nX ^ (x)|2 ' respec- 

n>\ ' 1>1 

tively. We shall denote the set of all trees avoiding the pattern set 2? 
by X7" no -^(X), and its subset consisting of trees with / leaves by 
■CTi, no -3*{X). The corresponding generating function is denoted by 
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fno-A z )' 

\-CT~l,no-,^(X)\z l 



(9) fno-Az) = £ 



l>\ 

Remark 1. One can also include the second variable in the generating 
series to count the internal vertices separately, and use the series 



(10) grxo-Az,t) = Y u 



U 

i,l>\ 

which in particular would make all the sets finite when internal ver- 
tices of our trees are allowed to have a single child (and the label set 
X includes Xi). To keep the exposition simple, we avoid discussing 
these subtleties here. 

The key feature of exponential generating functions in the con- 
text of planar X-labelled rooted trees is expressed by the following 
proposition. 

Proposition 2. Suppose that J(f and Jzf are two sets of planar X-labelled 
rooted trees. Let us define a set ^£ as follows: it consists of all trees T that 
have an occurrence of a tree pattern from J(T rooted at the root ofT, with the 
additional condition that all the subtrees rooted at the leaves of that pattern 
are occurrences of tree patterns from ££. Then 

(ii) fAz) = fAfAz)). 

Proof. Clearly, 

\X n ZTAX)\{fAz)) 1 



(12) fAfAz)) = Yj 



and it remains to note that the coefficient of z n in {fA z )) 1 is the 
number of ordered forests of I tree patterns from J2? with the total 

leaf set {1, . . . , n), therefore ^[ z)) can be thought of as the enumerator 
for forests satisfying the increasing condition for minimal leaves. □ 

2.2. Tree patterns and other types of consecutive patterns. In this 
section we assume, for simplicity, that X = X 2 (this corresponds to 
considering only binary tree patterns). 

Our first observation is that for |X| = 1 the set of all permuta- 
tions is naturally embedded in £T(X) as left combs: recall that a left 
comb has no conditions on where to put labels 2, 3 . . ., so left combs 
with n + 1 leaves are in one-to one correspondence with permuta- 
tions of length n. If we denote by T(o) the tree corresponding to 
the permutation a, then subtrees of T(cr) are in one-to-one correspon- 
dence with subwords of a, and the notion of a tree pattern for left 
combs corresponds precisely to the notion of a consecutive pattern 
for permutations. Moreover, if n is a set of consecutive permutation 
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patterns, and contains the left combs corresponding to elements 
of n and the right comb with three leaves, then the number of trees 
with n + 1 leaves that avoid is equal to the number of permu- 
tations of length n that avoid n. For |X| > 1, the same construction 
with left combs leads naturally to the notion of pattern avoidance in 
coloured permutations [29]. 

Moreover, the set of all words in a given alphabet A is naturally 
embedded in £7~{X) with X = X 2 = A as right combs. Indeed, re- 
call that for a right comb with n + 1 leaves there is exactly one way 
to label its leaves to fulfil the local increasing condition; to obtain a 
planar X-labelled rooted tree, it remains to label its internal vertices 
by A, and the ways to do so are in one-to-one correspondence with 
A-words of length n. If we denote by T(w ) the tree corresponding to 
the word w, then subtrees of T(w) are in one-to-one correspondence 
with subwords of w, and the notion of a tree pattern for right combs 
corresponds precisely to the notion of a divisor for words. Moreover, 
if W is a set of words, and &w consists of the right combs corre- 
sponding to elements of W and all the left combs with three leaves, 
then the number of trees with n + 1 leaves that avoid is equal to 
the number of words of length n without divisors from W. 

It is also possible to go the other way round and replace trees 
by objects resembling permutations and words. Let us assume, as 
above, that |X| = 1 (ignoring this technical assumption will, as al- 
ways, merely lead to coloured objects of the same sort). There is a 
very natural way to "straighten" the tree patterns and thus translate 
our questions into similar questions about patterns in sequences. Re- 
call that the total number of planar labelled rooted tree patterns in 
our case is equal to = (2n - 1)!!, the double factorial number. This 
number is also equal [14] to the number of permutations of the mul- 
tiset {1, 1, 2, 2, . . . , n, n} for which all the numbers appearing between 
the two occurrences of k are greater than k (for every k = 1, . . . , n). To 
a planar labelled rooted binary tree T, it is easy to assign recursively 
a permutation o(T) of that sort. For that, it is convenient to think of T 
as of a left comb with subtrees grafted in the places of right children 
of internal vertices. We denote those subtrees by T\, . . . , T^, in the 
order from the leftmost one to the rightmost one. Each subtree T; has 
its leaf labels belonging to a subset of {1, 2, ... , n\, so, strictly speak- 
ing, they are not trees of the sort we consider, but, as usual, we can 
identify them with planar labelled rooted trees via standardisation, 
so we may apply a to them, obtaining permutations of appropriate 
multisets. We assume that a takes the only one-vertex tree to the 
empty word, and put 



(13) 




for k > 2. 
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For example, o(^) = 11, o(\7 ) = 112332, o{ \y) = 122133, 




o(\/ ) = 122331; this leads to a meaningful notion of a generalised 

permutation pattern. It would be interesting to investigate this no- 
tion properly, in particular, to explore the links with patterns in set 
partitions [19, 25, 37], and also to see if the constructions of [10] can 
be adapted here. 



3. Asymptotic enumeration 

In this section, we discuss results on the asymptotic enumeration 
of trees avoiding a given set of patterns, where the results turn out 
to be in a way mimicking the results on the asymptotic enumeration 
for consecutive patterns in permutations [9, 11]. Our main tool is the 
following result, an adaptation of the classical Golod-Shaf arevich in- 
equality [17]; it is closely related to a similar inequality for symmetric 
operads [34]. 

Theorem 1. For every (possibly infinite) pattern set 0? , we have the fol- 
lowing coefficient-wise inequality of power series: 

(14) fAfno-Az)) ~ fx(fno-Az)) + fno-Az) > Z. 

Proof. Let us consider two series of finite sets: the set B n is the subset 
of JLT n (X) consisting of trees T whose subtrees rooted at the children 
of the root of T avoid patterns from & , and the set C„ is the subset of 
the set of pairs & x £,T„(X) consisting of all pairs (P, T) where there 
exists a subtree S of T rooted at the root of T for which st(S) = P, 
and all the trees rooted at the leaves of S avoid patterns from 
Proposition 2 implies that 

(is) Yj l ^r z " = Mf°»*W>' 

ri>\ 

(16) V ^Z" = fAfno-Az))- 

t—i n\ 

n>\ 

Therefore, our power series inequality translates into 

(17) |C„| - |B B | + \£T n , no .AX)\ >0,n>2. 

This follows from an observation that there exists an obvious surjec- 
tion from C„ U -CT„ /no -gg(X) onto B„: a tree from B„ either avoids , 
or has a pattern from & rooted at its root. □ 
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Corollary 1. Suppose that the power series h(z) = - ^ /ga(z) has non- 
negative coefficients. Then we have 

(18) ixr ^(X)| ^ i [zn - 1]h{z)n 

n\ n 

Proof. Let g{z) = z-fx(z)+fA z )- According to the Lagrange inversion 
formula [39], for the coefficients of the compositional inverse g { ~ l) {z) 
we have 

(19) [z n ]g { ~ l \z) = -[z"- 1 ] (4t) = -\z n - x \h{z)\ 

n \g{z)J n 

so under our assumption on h(z) the power series g { ~ x \z) has non- 
negative coefficients. According to Theorem 1, the power series 

(20) fAfno-Az)) ~ fx(fno-Az)) + fno-Az) = g( f no- Az)) 

has non-negative coefficients as well, so we see that 

(21) f no _Az) = g^Wno-Az))) = 

= Yj (l^Mz^giUAz))" > X {kz^Mzfy. 

n>\ n>\ 

This means that for every n > 1 we have 

\£T AX)\ ^ i [z n-i ]h{zT/ 

n\ n 

which is exactly what we want to prove. □ 

We use this result to prove the following theorem which gives a se- 
ries of examples when the set of ^-avoiding trees with n leaves grows 
rapidly, namely as C" times the number of all trees with n leaves for 
some constant C. A part of the proof is parallel to the corresponding 
proof in [9]. 

Theorem 2. Suppose that X = Xdfor some d>2, and that the power series 

(23) A(z) = 1 -E^l + M> 

a! z 

has a root a > 0. Then we have 

(24) \£T n , no -AX)\ > -^V- \£T n (X)\. 

\\X d \d-iaJ 

Proof. Recall that if X = X d , then the set £fT n {X) is non-empty only 
for n = kd - k + 1 = k(d - 1) + 1 for some k. Therefore, h(z) is a power 
series in z d ~ l ; we shall consider the series 

(25) h(t) = l-t + Y J a kt\ 

t>2 
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for which h(z) = h (^rZ 1 * -1 ); the only fact about the coefficients of that 
series that we use is that all the coefficients a k are non-negative (a k 
is a positive multiple of the number of certain labelled trees with k 
internal vertices). Under our assumption, this power series has a 
root j6 = l ^a d -\ 

Let us consider multiplicative inverse, £/>o bit 1 := (n(f)) -1 ; clearly, 
b = 1 and b„ - b n -\ + YX=i a $n-k - 0. Let us prove by induction 
that b n > j6 _1 fc„_i. Indeed, for n = 1 this statement is obvious (fi > 1 
because otherwise h(fi) is evidently positive), and forn > 1 we note 
that by the induction hypothesis b n -i > $ l ~ k b n - k , so 

n n 

b n = b n -i - akh-k > K-i - ^ a k p k - x b n -i > 



k=2 k=2 

( 

ok 



k>2 \ k>2 

and the statement follows. Hence the k th coefficient of (h(t))~ l is 
greater than or equal to /T fc . Therefore, the (k(d - l)) th coefficient 

of h(z) is greater than or equal to (^) • (^f^) = cC k{d - x) '. This 
means that 

(26) h(z) > g a^-Dz^-D = (l - J 1 , 

so consequently 



(27) "^(H^ 



Since the coefficients of h(z) are non-negative, Corollary 1 applies, 
and we deduce that 

(28) |XTfaj - fc+1 ' n °-^ (X)l > [z k ^]h(z) kd - k+1 > 

y ' (kd-k + l)\ ~ kd-k + V J y ' 

> 1 a -w-v( M -k + l + k-l\ 1 m -Jkd\ 
~ kd-k + 1 \ k ) kd-k + 1 \kj- 

This, in the view of Formula (4), simplifies to 

(29) \£T M - k+1 , no MX)\ >(kd-k + W M \ + l a ~ k(d ~ l) \^ = 

_ w _ x) (kdy. _ idi \ k (kdy _ ( di \ k 

which easily can be transformed into the inequality we want to prove. 

□ 
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This theorem, in particular, can be used to obtain estimates in the 
case of one tree pattern in the case |X| = 1, that is for trees with all 
internal vertices of the same arity and carrying the same label. 

Theorem 3. Suppose that |X| = 1, so that X coincides with a one-element 
set Xdfor some d>2, and that the set of forbidden patterns consists of one 
single pattern P with k > 2 internal vertices. Then for every pair (d, k) 
except for (2, 2), (2, 3), (2, 4), and (3, 2) there exists a positive number C 
(depending only on d and k but not on the actual pattern) such that 

(30) ixr„, no -p(x)i > c^ixt^x)!. 

Proof It is enough to show that for all pairs (d, k) except for (2, 2), 
(2,3), (2,4), and (3,2), the polynomial h(z) = 1 - ^ + ^-g^ has a 

positive root. Denoting, as above, t = we see that it is enough 
to prove that the polynomial h(t) = 1 - f + pq^jj has a positive root. 
For that, we note that the derivative -1 + of the polynomial 

h(t) has the only positive root t = f (faj ~^| 1)! V" 1 ; thus, to prove that 
h(t) has a positive root, it is enough to prove that its minimal value 
is attained at t and is negative. Using the formula t k ~ l = (fai ~^ 1)! , we 
see that h(t ) = 1 - f + j, so it suffices to prove that t > or 

m (*rf-fc + l)!(fc-l) fc - 1 „ 1 

which is true in all cases we consider, since it is true for (d, k) = (2, 5), 
(d, k) = (3, 3), and (d, k) = (4, 2), and its left hand side increases if 
either d or k increases. □ 

Our results suggest a new numerical invariant of a set of patterns: 
Definition 3. The growth rate of a set of patterns s c £T{X) is 

'\£T n , no MX)\\^ 



(32) lim sup 



\£T n (X)\ 



We expect that the methods of this section can be easily generalised 
to prove that for every set of labels X (and for some k depending on 
X) every pattern in £fT{X) with at least k internal vertices has positive 
growth. Moreover, computer experiments suggest that the follow- 
ing conjecture generalising Warlimont's conjecture for consectuive 
patterns [41] (proved recently by Ehrenborg, Kitaev and Perry [10] 
via a beautiful link between consecutive pattern avoidance and the 
spectral theory for integral operators on the unit cube) holds for tree 
patterns. 



PATTERN AVOIDANCE IN LABELLED TREES 



13 



Conjecture 1. For every set of labels X, there exists an integer d that 
for every pattern P in £T(X) with at least d internal vertices and some 
numbers c(P) > 0, A(P) > live have 

(33) \£T n {X)\ ~ C(P)A(P) • 

It would be most interesting to adapt the approach of [10] for tree 
patterns; such an adaptation, in addition to its consequences for the 
asymptotic enumeration questions, may be of substantial interest for 
the operad theory as well. Another natural question arising from the 
asymptotic enumeration is to understand the "growth hierarchies" 
of tree patterns, using the strategy of [26] or otherwise. 



4. Exact enumeration: cluster inversion formula 

The following result on power series inversion is simultaneously 
a generalisation of the inversion formula for planar tree patterns 
[1, 28, 33] and of the cluster inversion formula of Goulden and Jackson 
for words and permutations [20, 32]. This result is proved in [7] by 
simple homological algebra; we formulate it here in a different way, 
so that an interested reader will easily prove it directly using the 
inclusion-exclusion formula, similarly to the usual cluster inversion. 

Definition 4. Let 2? be a set of patterns in £T(X). A tree T together 
with a collection of its subtrees Ti,...,T k is said to be a k-cluster if the 
following two conditions hold: 

(1) For every i, the subtree is an occurrence of a pattern from 
&>: st(T f ) e 

(2) for every edge e of T that joins two internal vertices, it is an 
edge between the two internal vertices of some T,-. 

By definition, the set of 0-clusters is the set of labels X. 

Informally, a fc-cluster is a tree which is completely covered by k 
copies of patterns from s . 

Let us denote by c n ^{^) the number of fc-clusters for which the 
tree T has n leaves. 

Theorem 4 ([7]). The compositional inverse for f no -?A z ) can be computed 
via clusters as follows: 

(34) E ^ 

n>l,fc>0 

The following consequence of the general inversion formula is 
a direct analogue for planar labelled tree patterns of the inversion 
formula [1, 28, 33] for planar tree patterns. 
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Corollary 2. Suppose that X = X 2 , and that 3* and £? are two com- 
plementary sets of tree patterns with 3 leaves, U £? = £Ts(X). The 
corresponding generating functions for pattern avoidance are inverse to 
each other: 

(35) fno-,9>{- fno-J2{-Z)) = fno- M~ j no- = z - 

Proof It is easy to see that in this case fc-clusters for are in one-to- 
one correspondence with trees that avoid £ (each such tree admits 
the unique covering by patterns from and that the signs in the 
inversion formula match those suggested by (35) (since in our case 
the underlying tree of every n-cluster has n + 2 leaves). □ 

Note that for left (right) combs corresponding to some consecutive 
permutation patterns (words), clusters for left combs are the left 
(right) combs corresponding to the usual Goulden-Jackson clusters 
for these patterns (words). This instantly proves the following result. 

Corollary 3. Suppose that two sets of consecutive permutation patterns 
(words) are Wilf equivalent. Then the two sets of tree patterns consisting 
of the left (right) combs corresponding to the given permutation patterns 
(words) are Wilf equivalent as tree patterns. 

As in the case of permutations, we expect that at least in the case of 
a single pattern a careful study of its self-overlaps ("overlap sets" of 
[23], or equivalently "overlap maps" of [31]) would be very beneficial 
for studying Wilf equivalence. We shall discuss it in detail elsewhere, 
mentioning a particular case briefly in the next section. 

Let us conclude this section with an open question. For consecu- 
tive patterns in permutations, Mendes and Remmel developed the 
symmetric functions method [30] for enumeration of permutations 
avoiding the given set of patterns. It is natural to expect that this 
method can be generalised to deal with the case of tree patterns, pos- 
sibly making use of the plethysm for symmetric functions where our 
formulas compute compositions of power series. Some inversion for- 
mulas in the completion of the algebra of symmetric functions exist 
in the operadic context, being provided by homological algebra, in 
particular the operadic Koszul duality for symmetric operads [16], 
however, once we move from abstract trees to their representatives 
(that is, planar labelled rooted trees studied in this paper), there is no 
clear way to incorporate symmetric functions in the picture. 

5. Examples 

5.1. Case X = X 2 , \X\ = 1. In this section, we assume that X = X 2 , 
|X| = 1, that is, we only work with binary trees, and do not use labels 
for internal vertices. To simplify the notation, we suppress X in the 
formulas, and write simply £T n etc. 
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We begin with classifying the Wilf classes of pattern sets with three 
leaves. 

Theorem 5. There exist exactly four Wilf classes of sets of pattern sets with 
three leaves. 

Proof This theorem follows from the two lemmas which give a pre- 
cise description of the five Wilf classes. 



13 12 2 3 



Lemma 5.1. The three patterns 




are Wilf equivalent to 



each other; the number of trees with n leaves avoiding either of them is equal 
to (n - l)\for each n>3. 

13 12 2 3 

\/ 2 \/ 3 i \/ 

Proof Let us denote Pi = \Z, P 2 = x/, and P 3 = \y . 

A correspondence p between the set of Pi -avoiding trees and the 
set of P2-avoiding trees can be defined recursively as follows. By 

12 12 

definition, = v . Let us represent a tree T £ -CT n ,no-Pi as a 

left comb with some subtrees T\,...,T k (listed along the way from 
the root) grafted at its "right-looking" leaves. To determine p(T), we 
apply p to the subtrees T,- and reverse the order of grafting. In other 
words, we graft p(T k ) in the place of T\, p(TY_i) in the place of T 2 etc. 
Clearly, p that identifies -CT~ n>no - Pl with JLT„ A0 .p 2 . 

A correspondence k between the set of Pi -avoiding trees and the set 
of P 3 -avoiding trees can be defined recursively as well. By definition, 

12 12 

P( X/ ) = v - Let us represent a tree T e -LT nA0 . Pl as a left comb with 
some subtrees T\,...,T k (listed along the way from the root) grafted 
at its right-looking leaves. We note that the set -CT„ rno -p 3 is the set 
of all left combs with n leaves; by induction we may assume that 
we already know the left combs k(Ti), . . . , x{T k ). Let k(T) be the 
left comb whose right-looking leaves, listed along the way from the 
root, are the right-looking leaves of K(T k ), the right-looking leaves of 
k(Ta:-i)/ • • • / the right-looking leaves of k(Ti). The observation (which 
can easily be proved by induction) that the label of the right-looking 
leaf of k(T) which is the farthest from the root is the equal to the 
smallest leaf label of T\ shows how to construct the inverse of k, so 
we identified JLT„ /no . Pl with -CT„ ino .p 3 . 

In addition, since the set £fr nA0 . Pz is the set of all left combs with n 
leaves, it has the cardinality (n - 1)!, so for each of the three subsets 
& c £T 3 with |^| = 1 and for each n, there are exactly (n - 1)! 
different ^-avoiding tree with n leaves. □ 
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Lemma 5.2. The three two-pattern sets { V, \/}, { \/, \/ }, and 



3 

2 



2 

2 1 



{ \y, \y } are Wilf equivalent to each other; the number of trees with n 
leaves avoiding either of these sets is equal to lfor each n>3. 

12 13 

Proof. Indeed, for & = {"Xy, ^X/} the only ^-avoiding tree with 

12 2 3 

n leaves is the only right comb, for & = C^Cy, \y // } the only 0#- 

avoiding tree with n leaves is the only left comb with labels of right- 
looking leaves increasing along the way from the root, and for & = 



3 2 
2 1 



{ V / , \/ } the only ^-avoiding tree with n leaves is the only left 

comb with labels of right-looking leaves decreasing along the way 
from the root. Therefore for each of the three subsets 2? c XT3 with 
\&\ = 2 and for each n, there is exactly one ^-avoiding tree with n 
leaves. 

Alternatively, one can apply the inversion formula (35): from 
Lemma 5.1, we conclude that the exponential generating function 
for every one-pattern set is equal to -log(l - z); computing its in- 
verse and adjusting the signs instantly shows that the exponential 
generating function for every two-pattern set is exp(z) - 1, which is 
the exponential generating function for the sequence 1, 1, 1, . . .. □ 

Since the empty pattern set and the pattern set containing all trees 
with three leaves form their own Wilf classes, the theorem follows. 

□ 



For pattern sets with at least four leaves, we have only partial 



results. Note that there are 15 patterns with four leaves: 




12 13 13 14 14 232434 




2 3 2 4 3 4 




Therefore the number of sets of 



tree patterns with 4 leaves is equal to 2 5 = 32768, so a complete 
classification of Wilf classes is already a very heavy task. We shall 
present a very simple result on classification on Wilf classes for sets 
consisting of a single pattern. 
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Theorem 6. There exist exactly five Wilf classes of sets of one pattern with 
four leaves. 

Proof This theorem follows from a sequence of lemmas which give 
a precise description of the five Wilf classes. 



12 14 3 4 

3 \/ 3 2 

4 \/ 2 . 1 





Lemma 6.1. The three patterns X/, \/, and \y are Wilf 
equivalent to each other. 

Proof. There is a bijective proof which is completely analogous to that 
of Lemma 5.1; we leave it to the reader to fill in the details. □ 

3 4 2 3 2 4 

1234132414231 \/ \/ 4 \/ 3 

2 1 



Lemma 6.2. The six patterns \/ , \/ , \y , \/, \/ , and 
are Wilf equivalent to each other. 




3 4 

1234 1324 1423 1 

2 



Proof. Let us denote Pi = \/ , P 2 = \y , P 3 = \/ , P 4 = 




2 3 2 4 

P 5 = \y , and P 6 = 



Let us show that P\ ~ w P2 by exhibiting a one-to-one correspon- 
dence a between the Pi-avoiding patterns and the P 2 -avoiding ones. 
If a tree T avoids both Pi and P 2 , we put a(T) = T. If T avoids Pi 
but contains P 2 , we may assume that there is an occurrence of P 2 at 
the root of T (otherwise we find the internal vertices closest to the root 
that are roots of occurrences of P 2 , and apply a recursively at these 



vertices). Let T = \y , and denote S; = a(Ti), i = 1, . . . , 4. We put 



aCT) = \/ . We constructed a bijection between Pi-avoiding trees 

containing P 2 and P 2 -avoiding trees containing Pi. The case of Pi and 
P3 is handled in a similar way. 

Let us show that Pi ~ w P4 by exhibiting a one-to-one correspon- 
dence j6 between the Pi -avoiding patterns and the P 4 -avoiding ones. 
If a tree T avoids both Pi and P4, we put /3(T) = T. If T avoids Pi 
but contains P4, we may assume that there is an occurrence of P4 
at the root of T (otherwise we find the internal vertices closest to the 
root that are roots of occurrences of P4, and apply /3 recursively at 



these vertices). Let T = \/ , and denote Si = j8( V ), S 2 = j8( ^ 4 ). 

Si Sj 

We put j6(T) = Note that the only vertex of this tree where an 
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occurrence of P4 can be rooted is the root. However, if there is an 
occurrence of P4 there, it is easily seen to imply an occurrence of Pi in 
T, a contradiction. We constructed a bijection between Pi-avoiding 
trees containing P2 and P2-avoiding trees containing Pi. 

The equivalence P 5 ~ w P 6 can be established inductively similarly 
to how it is done in Lemma 5.1. 

Finally, the easiest way to see the equivalence Pi ~ w P 5 is via the 
inverse generating functions. Basically, Pi and P5 have the same 
structure of self-overlaps: there are two self-overlaps, one of which 
is "rigid" (only one labelling of leaves is consistent with the local 
increasing condition), and the other one admits three different leaf 
labellings. This allows for an inductively constructed bijections be- 
tween the clusters that control the coefficients of the inverse series. 
We leave the details to the reader. □ 





Lemma 6.3. The four patterns \/, \/, \/ , and \y are 
Wilf equivalent to each other. 

Proof. The corresponding permutation patterns are Wilf equivalent, 
so the Corollary 3 applies. □ 

Combining the lemmas above with a somewhat lengthy computa- 
tion showing that 

• for the class described in Lemma 6.1 the sequence count- 
ing the trees avoiding the patterns of that class begins with 
1,1,3,14,91,756,7657, 

• for the class described in Lemma 6.2 the sequence count- 
ing the trees avoiding the patterns of that class begins with 
1,1,3,14,90,739,7392, 

• for the class described in Lemma 6.3 the sequence count- 
ing the trees avoiding the patterns of that class begins with 
1,1,3,14,90,738,7364, 




for the pattern \y the sequence counting the trees avoiding 
that pattern begins with 1, 1, 3, 14, 90, 740, 7420, 



• for the pattern \y the sequence counting the trees avoiding 

that pattern begins with 1, 1, 3, 14, 90, 737, 7336, 
we conclude that there are exactly five Wilf classes. □ 

Of the five integer sequences we saw in the previous proof, only 
two seem to appear in the Online Encyclopedia of Integer Sequences 
[38]: the third one matches A088789, the sequence of coefficients in 
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the compositional inverse of the power series 



2x 



, and the first one 



l+exp(x) 



matches A183611, which, if we take care of the difference in number- 
ing, is described as the sequence of coefficients of the power series f(z) 
satisfying the differential equation f"{z) = f'(z) 2 + zf'(z) 3 . The first 
of these descriptions is not too surprising, as the inversion formula 
(34) suggests that the inverse series makes lots of sense combinato- 
rially. The second description is related to the results of Khoroshkin 
and Piontkovski [22] who proved that in some cases the generating 
function does indeed satisfy a differential equation; however, the 
patterns of that Wilf class are not covered by their results, so the 
appearance of a differential equation in this enumeration problem 
would be another bit of evidence supporting their general conjec- 
ture that states that for every finite set of patterns the corresponding 
generating function satisfies a non-linear differential equation with 
polynomial coefficients. 

It is natural to ask which tree patterns are "the hardest to avoid", 
that have the fewest numbers of trees that avoid them, and which tree 
patterns are "the easiest to avoid", that is have the largest numbers 
of trees that avoid them. After examining the proof of the previous 
theorem and performing some computer experiments, we arrived 
at the following conjecture which is closely related to the conjecture 
of Elizalde and Noy [12] that the permutation 12 ... n (the identity 
permutation) is the easiest to avoid, and to the conjecture of Naka- 
mura [31] that the permutation 12 ... (n - 2)n(n - 1) (the transposition 
of its last two entries) is the hardest to avoid. 

Conjecture 2. Let us denote by LC<, LC>, and RC„ the left comb with n 
leaves whose leaf labels increase along the way from the root, the left comb 
with n leaves whose leaf labels decrease along the way from the root, and the 
right comb with n leaves respectively. The patterns LC<, LC>, and RC n are 



5.2. Case X = X 2 , |X| = 2. Throughout this section we assume that 
we work with binary trees with two possible labels for internal ver- 
tices, in other words, X = X 2 = {o, •}. 

The following result is still easy to obtain "by hand". 

Theorem 7. There exist exactly two Wilf classes of sets of one pattern with 
three leaves, and exactly ten Wilf classes of sets of two patterns with three 
leaves. 

Proof. First of all, let us note that because of the inversion formula (35), 
we can work with the trees avoiding 11 and 10 patterns respectively. 
For the avoidance of 11 patterns, the only allowed pattern can either 
use only one internal vertex label (in which case Theorem 5 means 
that there exists only one allowed tree for each number of leaves) or 



the easiest to avoid, and the pattern 



v 



is the hardest to avoid. 
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use two different internal vertex labels (in which case there is no way 
to build an allowed tree with 4 or more leaves). 

For the avoidance of 10 patterns, our theorem follows from a se- 
quence of lemmas which give a precise description of the ten Wilf 
classes. Lemmas 7.1-7.7 are almost obvious, since the trees that are 
being counted admit very explicit descriptions; we omit their proofs. 
Note that we switch to complements again: instead of listing the ten- 
element sets, we list their complements in the set of all tree patterns 
with three leaves. 

Lemma 7.1. The six sets whose complements in the set of all tree pat- 

12 13 13 23 12 23 

\y 3 \y 2 \y i i \/ \y 3 i \y 
terns with three leaves are { \/, \/}, { \/,\/ }, { \/, \/ }, 

1213 13 23 12 23 

3 \,/ 2. ,\/ 2 1 \/. \/ 3 1 



{ \/, \/}, { \/, V }, { \/,\/° } are Wilf equivalent to each 

other; the number of trees with n leaves avoiding either of these sets is equal 
to (n - l)\for each n > 4. 

Lemma 7.2. The two sets whose complements in the set of all tree patterns 

12231223 

\/ 3 1 \/ \/ 3 1 \/ 

with three leaves are { \/, \/ }, { \/, \/° } are Wilf equivalent to 

each other; the number of trees with n leaves avoiding either of these sets is 
equal to § (f )\for even n>4and to (^)l + \ {^f)\for odd n>3. 

Lemma 7.3. The ten sets whose complements in the set of all tree pat- 

1213 1312 1213 

■ , , , rV 3 V 2 > , V 2 V 3 , , V 3 V \ 

terns with three leaves are { \/, \/}, { \/, \./}, { \./, \/\, 

1312 1213 1312 1213 1312 

\/ 2 \/ 3 \/ 3 \/ 2 \/ 2 \/ 3 \/ 3 \/ 2 \/ 2 \/ 3 

2312 2312 

3. .1 V/ \/ 3. 



{\./ , \/}, {\/ , \/} are Wilf equivalent to each other; the number 

of trees with n leaves avoiding either of these sets is equal ton - lfor each 
n > 4. 

Lemma 7.4. The 32 sefs whose complements in the set of all tree pat- 

1212 1213 12 23 

\y 3 \y 3 \y 3 \/ ^ \y 3 i \/ 
terns with three leaves are { \./, \/}, { \./, \/}, { \./, \/ }, 

1312 1313 13 23 2312 2313 

.\./ 2 \ / 3. \y 2 \/ 2. ,\./ 2 1 \/. .1 \/ \/ 3. .1 \./ \/ 2. 



23231212 1313 1212 1313 

r 1 \/ 1 \/, ,\/ 3 \/ 3 , ,\/ 2 \/ 2 , ,\/ 3 \/ 3 , ,\/ 2 V 2 , 

2323 2323 23231313 1212 

1 \/ 1 \/ 1 \/ 1 \/, 1 \/ 1 \/, r V 2 \/ 2 \/ 3 \/ 3 

{V /\/ }/{\/ ,\/ }/{\/ /V }/{ V/ V / },{ N// N/L 
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1313 2323 23231212 1313 

{ \/, \/},{V /V /V K \// Vil \/, N/L 

2323 2313 2313 2312 2313 

2. 1 \/ V/ 2. 1 \ / \/ 3. 1 



{\/° ,v M\/ , \/M\/ , VUv / \/}/{\/ / vL 

12 2313 2313 2313 23 

\/ 3 1 \/ \/ 2 1 \/ \/ 2 1 \/ \/ 2 1 \/ 

{ N//V }/ V/V }/{ V, V }/{ V,V° } are Wilf equiva- 
lent to each other; the number of trees with n leaves avoiding either of these 
sets is equal to 2 for each n > 4,. 

Lemma 7.5. The two sets whose complements in the set of all tree patterns 

12231223 
\/ 3 1 \/ \/ 3 1 \/ 

with three leaves are { \/, \/ }, { \/, \/° } are Wilf equivalent to 

each other; the number of trees with n leaves avoiding either of these sets is 
equal to lfor n = 4, and equal to Ofor each n > 5. 

Lemma 7.6. The two sets whose complements in the set of all tree patterns 

13231323 

' ' 2 i \y_ \y 2 i 



with three leaves are { \/,\/' }, { \/,\/° } are Wilf equivalent to 

each other; the number of trees with n leaves avoiding either of these sets is 
equal to 2 for n = 4, and equal to Ofor each n>5. 

Lemma 7.7. The two sets whose complements in the set of all tree patterns 

12 13 12 13 

\/ 3 \/ 2 \/ 3 \/ 2 

with three leaves are { \/, \/}, { V, \./} are Wilf equivalent to 

each other; the number of trees with n leaves avoiding either of these sets is 
equal to Ofor each n > 4. 

Lemma 7.8. The six sets whose complements in the set of all tree pat- 

12 2313 2312 23 

3 1 \_/. ,\./ 2 1 \/. ,\/ 3 1 



terns with three leaves are { \/, \./° }, { \/, \/° }, { \/ , \/ }, 

13 23 2313 2313 

\/ 2 1 \/ 1 \/ \/ 2 1 \/ \/ 2 

{ \/, \/ }, {\/ , \/}, {\/° , \/} are Wilf equivalent to each 

other; the number of trees with n leaves avoiding either of these sets is equal 
to the number of involutions in S n -\for each n > 4. 

Proof Examining the structure of the allowed trees, we see that in 
each case we have a right comb or an increasing (decreasing) left 
comb, possibly with leaves replaced by pairs of leaves. Examining the 
leaf labels, one easily extract a decomposition of an involution into the 
product of disjoint cycles, which gives a one-to-one correspondence. 

□ 

Lemma 7.9. The two sets whose complements in the set of all tree patterns 

2312 ' 2312 

with three leaves are {V^, \/} and {\/ o/ , ^\/} are Wilf equivalent 
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to each other; the number of trees with n leaves avoiding either of these sets 
is equal to the n th Fibonacci number (f = 0, f = 1, / w+1 = f„+ f n -i)for 
each n>3. 

Proof Examining the structure of the allowed trees, we see that in 
each case we have a right comb, possibly with leaves replaced by 
pairs of leaves, but with the leaf labels increasing globally along the 
path from the root, therefore the number of trees we are trying to 
compute is equal to the number of sequences of l's and 2's that sum 
up to n - 1, which is known to be the Fibonacci number [39]. □ 

Lemma 7.10. The two sets whose complements in the set of all tree patterns 

13 12 1312 
\/ 2 \/ 3 \/ 2 \/ 3 

with three leaves are { \/ , \/}, { \/, \./} are Wilf equivalent to 

each other; the number of trees with n leaves avoiding either of these sets is 
twice the number of alternating permutations in S n -\for each n>3. 

Proof Clearly, each allowed tree is a left comb, the labels o, • along 
the path from the root alternate, and so do the leaf labels. Therefore, 
the statement is obvious: alternating permutations come from the 
leaf labels, and "twice" reflect the fact that for each k there are two 
alternating sequences of o's and »'s of length k. □ 

□ 

Further (computer-aided) investigation shows that the following 
statement is true: 

Theorem 8. For sets of patterns with 3 leaves, there are 

(1) 2 Wilf classes of 1-element sets, 

(2) 10 Wilf classes of2-element sets, 

(3) 40 Wilf classes of3-element sets, 

(4) 99 Wilf classes of ^-element sets, 

(5) 189 Wilf classes of 5-element sets, 

(6) 202 Wilf classes of 6-element sets, 

(7) 189 Wilf classes of 7 -element sets, 

(8) 99 Wilf classes of 8-element sets, 

(9) 40 Wilf classes of9-element sets, 

(10) 10 Wilf classes oflO-element sets, 

(11) 2 Wilf classes of\\-element sets. 

To conclude this section, let us mention a promising direction 
towards new bijective proofs for enumeration of Wilf equivalence 
classes. In the case of unlabelled planar trees, many bijections have 
been constructed in [13], where the "word notation" for trees was 
used. That "word notation" is a construction of crucial importance 
for the operad theory. It was discovered by Hoffbeck [18] who de- 
fined a partial ordering compatible with the operad structure on 
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the set of "tree monomials" which was then used to formulate and 
prove a PBW criterion for Koszul operads. Later, this partial or- 
dering was extended to a total ordering (still compatible with the 
operad structure) in [8]; that ordering is one of the key ingredients 
of the Grobner bases machinery in the case of operads. The latter 
total ordering is coming from replacing a tree T 6 £.T{X) with a 
pair (path(T), perm(T)) consisting of a "path sequence" path(T) (a se- 
quence of words in the alphabet X) and a permutation perm(T) of 
the set of leaves of T. All questions of pattern avoidance for trees 
can be translated into questions of pattern avoidance for this type of 
data; the corresponding notion of a pattern is a certain mixture of the 
classical notion of a divisor of a word and the notion of a generalised 
pattern in permutations [2, 3] (different from the naive notion of a 
generalised pattern in coloured permutations). This way of thinking 
about patterns in trees should be useful for bijective proofs; we hope 
to address it in more detail elsewhere. 



Appendix A. Shuffle operads and patterns in trees 

In this appendix, we recall some relevant definitions of the operad 
theory, and explain how the notion of pattern avoidance in trees 
arises naturally in this context. 

Let us denote by Ord the category whose objects are non-empty 
finite ordered sets (with order-preserving bijections as morphisms). 
Also, we denote by Vect the category of vector spaces (with lin- 
ear operators as morphisms). It is usually enough to assume vec- 
tor spaces finite-dimensional, though sometimes more generality is 
needed, and one assumes, for instance, that they are graded with 
finite-dimensional homogeneous components. 

Definition 5. (1) A (non-symmetric) collection is a contravariant 
functor from the category Ord to the category Vect. We shall 
refer to images of individual sets as components of our collec- 
tion. 

(2) Let & and be two non-symmetric collections. The shuffle 
composition product of and £1 is the non-symmetric collec- 
tion & o sh £1 defined by the formula 



(^ o sh J2)(J) : = &>(k) <8> ® • • • ® (*)) 

k [f : 

where the sum is taken over all shuffling surjections /, that is 
surjections for which min/ _1 (z) < min/ _1 (;) whenever i < j. 
(3) A shuffle operad is a monoid in the category of non-symmetric 
collections equipped with the shuffle composition product. 
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This definition is a counterpart of a more classical one, where one 
deals with finite sets without a specified order, and all surjections 
are allowed to define the composition. The corresponding monoids 
are called (symmetric) operads, and those are monoids widely used 
in algebra and topology, since they capture the algebraic properties 
of compositions of operations with several arguments. In particular, 
algebras of a certain type, like all associative algebras, or all Lie 
algebras, can be viewed as modules over a monoid of this kind 
(formed by all operations that can be defined on algebras of the 
given type), and this point of view proves to be useful. However, 
for computational purposes the fact that finite sets have symmetries 
gets in the way, and it turns out that shuffle operads allow to deal 
with some of the troubles arising because of that. (Every symmetric 
operad can be viewed as a shuffle operad, since every ordered set 
can be viewed as an unordered set.) 

Monoids in the monoidal category of vector spaces (with the usual 
tensor product) are associative algebras. One can present associative 
algebras via generators and relations; if all relations are monomials 
in generators, the corresponding algebra admits a straightforward 
basis consisting of all monomials avoiding the monomials from the 
set of relations. In general, there is no such elegant description; to 
obtain it, one uses the machinery of Grobner bases. A Grobner basis 
is a special choice of a set of relations that allows to find a "monomial 
replacement" for the given algebra A, that is an algebra with the 
same generators and with monomial relations for which the natural 
monomial basis is also a basis for A. Such a set of monomial relations 
is provided by the "leading terms" of the relations forming a Grobner 
basis. 

It is very natural to try and find an appropriate Grobner bases 
theory for operads. For operads with symmetries, it turns out to be 
impossible: not every symmetric operad has a monomial replace- 
ment. However, for shuffle operads it turns out to be possible. Let us 
be a little bit more precise. Similarly to the case of associative alge- 
bras, a shuffle operad can be presented via generators and relations, 
that is as a quotient of the free operad J^"(V), where V is the space 
of generators (which itself is a non-symmetric collection). If X is the 
collection of ordered bases for components of V, that is a functor 
from Ord to Ord, then the free shuffle operad generated by V admits 
a basis of "tree monomials" which can be defined combinatorially; 
they are precisely planar X-labelled rooted trees studied throughout 
this paper. Any shuffle composition of tree monomials is again a 
tree monomial. The crucial feature of shuffle operads is that they 
admit good monomial orderings (and therefore one can talk about 
leading terms of relations). More precisely, there exist several ways 
to introduce a total ordering of tree monomials in such a way that all 



PATTERN AVOIDANCE IN LABELLED TREES 



25 



the shuffle compositions respect that total ordering: increasing one 
of the monomials we compose increases the result. Furthermore, the 
algebraic statement that a tree monomial T is obtained from another 
tree monomial S by shuffle compositions (that is, T is divisible by S 
in our monoid) means, in the combinatorial language, that S occurs 
in T as a pattern. 

We now have all the ingredients to relate questions about shuffle 
operads to pattern avoidance in trees. A Grobner basis of an ideal I 
in the free shuffle operad is a system G of generators of I for which 
the leading monomial of every element of I is divisible by one of the 
leading terms of elements of G. Such a system of generators allows to 
perform "long division" modulo I, computing for every element its 
canonical representative. Tree monomials avoiding the leading terms 
of elements of G ("normal monomials") form a basis in the quotient 
by the ideal I; in other words, shuffle operads do admit monomial 
replacements. Thus, it is clear that enumerating trees avoiding the 
given set of patterns literally corresponds to computing dimensions 
of components of operads presented by generators and relations. 
(Of course, the first step is to compute a Grobner basis, see [8] for an 
algorithm approaching this problem.) From this point of view, the 
asymptotic enumeration is also very meaningful: in fact, the area of 
the operad theory primarily concerned with quotients of the operad 
describing associative algebras (better known as "theory of varieties 
of associative algebras") has asymptotic questions ("codimension 
growth") among its core questions [15, 21]. There were even some 
attempts to use the Grobner bases formalism in that context, see for 
example [27]. We expect that our approach will prove useful in that 
context. 
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